Introduction to Bayesian Statistics

Murray Logan

28 July 2024

Frequentist vs Bayesian

Frequentist

P(D|H)
long-run frequency
relatively simple analytical methods to solve roots
conclusions pertain to data, not parameters or hypotheses
compared to theoretical distribution when NULL hypothesis is true
probability of obtaining observed data or MORE EXTREME data

Frequentist

P-value
- probabulity of rejecting NULL
- NOT a measure of the magnitude of an effect or degree of significance!
- measure of whether the sample size is large enough
95% CI
- NOT about the parameter it is about the interval
- does not tell you the range of values likely to contain the true mean

Frequentist vs Bayesian

-----------------------------------------------------
                Frequentist          Bayesian  
--------------  ------------         ----------------
Probability     Long-run frequency   Degree of belief
                $P(D|H)$             $P(H|D)$

Parameters      Fixed, true          Random, 
                                     distribution

Obs. data       One possible         Fixed, true

Inferences      Data                 Parameters

-----------------------------------------------------

Frequentist vs Bayesian

n: 10
Slope: -0.1022
t: -2.3252
p: 0.0485

n: 10
Slope: -10.2318
t: -2.2115
p: 0.0579

n: 100
Slope: -10.4713
t: -6.6457
p: 1.7101362^{-9}

Frequentist vs Bayesian

	Population A	Population B
Percentage change	0.46	45.46
Prob. >5% decline	0	0.86

Bayesian Statistics

Bayesian

Bayes rule

\[ \begin{aligned} P(H\mid D) &= \frac{P(D\mid H) \times P(H)}{P(D)}\\[1em] \mathsf{posterior\\belief\\(probability)} &= \frac{likelihood \times \mathsf{prior~probability}}{\mathsf{normalizing~constant}} \end{aligned} \]

Bayesian

Bayes rule

\[ \begin{aligned} P(H\mid D) &= \frac{P(D\mid H) \times P(H)}{P(D)}\\ \mathsf{posterior\\belief\\(probability)} &= \frac{likelihood \times \mathsf{prior~probability}}{\mathsf{normalizing~constant}} \end{aligned} \]

The normalizing constant is required for probability - turn a frequency distribution into a probability distribution

Estimation: Likelihood

\(P(D\mid H)\)

Bayesian

conclusions pertain to hypotheses
computationally robust (sample size,balance,collinearity)
inferential flexibility - derive any number of inferences